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@ Procedure for decoding an audio signal In which other Information has been Included in said 
audioslgnal by making use of masking effect 



@ A property of the hearing system, i.e. masking effect, can be utilized, as Is known in the art, so that to 
any audio signal a second, weaker signal can be added inaudibly. In a system of prior art employing said 
principle an analysis of the audio signal and calculation of the masking threshold are carried out both in 
the transmitter snd the receiver. This operation results in a complicated decoder and on certain 
occasions, audible distortion in a repeated audio signal. According to the present invention this can be 
so avoided that the transmitter transmits all data required in decoding on a separate data channel. 
IHereby, the decoder operates under the control of the transmitter. 
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The present inventfon relates to transmission and 
reception of an audio signal when the audio signal is 
coded in a transmitter to be an appropriate signal for 
the transmission path so that digital information is 
hidden into the signal by making use of the masking s 
effect of the human ear. 

Multichannel sound has today become more gen- 
eral as the film sound because of the higher listening 
enjoyment offered thereby than the 2.channel stereo 
sound. Therefore, it Is necessary that also on the io 
HDTV that the 4-channel film sound could be repeat- 
ed as genuinely as possible. The currently used tel- 
evision systems, including HOTV, have, as regards 
the audio transfer, been however specified on a 2- 
channel stereo sound in which the bandwidth is not 15 
as such sufficient for transmission of a multichannel 
sound signal. Therefore, a 4-channel sound signal 
should in oneway or another be so coded that it is ap- 
propriate for transmission via the transfer path of a 2- 
channel stereo signal. In addition, the encoding 20 
should be carried out so that the received signal can 
be listened to as such with receivere currently used 
in the form of 2-channel stereo. One of such coding 
methods is presented in a conference proceeding 
Proc. ICASSP 90. Alberquerque, New Mexico. April 3- 25 
6. 1990. p. 1097-110. W. ten Kate. L van der Kerkhof 
and F. Zijderveld: Digital Audio carrying extra infor- 
mation. Said encoding method developed by Philips 
is described below in greater detail. 

Said encoding method makes use of two charac- 30 
teristic properties of human hearing: hearing thresh- 
old and masking effect. The masking effect means 
that to any audio signal can be added another, weak- 
er signal which is not audible to the human ear be- 
cause of the masking effect The masking effect is a 35 
psycho-acoustic effect In which the hearing threshold 
shifts upwards when other, lower sounds are present. 
The masking effect is best In sounds in which the 
specbrum components are close to t he components of 
the masking sound. The frequency mask becomes 40 
more rapidly weaker in changing towards lower 
sounds. This is true also regarding the time level: the 
masking effect is greatest In sounds which are heard 
simultaneously. The dependence of the masking ef- 
fect on time and frequency is well known in simple sig- 45 
nals. The existence of the masking effect can be 
made use of so that signals below the hearing thresh- 
old can be added to the audio signal. This happens in 
principle so that an audio signal is sampled and in the 
locations of the sample bits not audible to the human so 
ear other Information is placed. The information Is 
thus positfoned in place of the less signlf fcant bits of 
the sample which is in digital form. When a signal like 
this is repeated, the human ear is not at all able to 
hear the added signal because the signal actually in- 55 
tended to be heard masks it The masking capacity of 
the human ear therefore determines how many of the 
less significant bits can be substituted wit hout still be- 



ing audible. The added signal can be utilized for vari- 
ous purposes. SimHarly, when a sound signal is com- 
pressed, the signals below the hearing threshold can 
be entirely omitted or saved, or only those audio sig- 
nals can be transferred which are audible to the hu- 
man ear. The principle of said encoding method 
known in the art making use of the masking effect is 
presented in Fig. 1. An entering audio signal is sam- 
pled and divkJed f iret into a plurality of subbands in a 
f nter bank 1. and the signal samples of the subbands 
are dedmated in a member 2. The subbands are pre- 
ferably of equal size so that the sampling frequency 
according to the Nyqvisfs criterion in the decimation 
member 2 of each subband is the same. The samples 
of each subband are then grouped into consecutive 
time windows in member 3. The length of a time win- 
dow Is AT and it includes samples at a given point of 
time from each subband. Thus, the simultaneous time 
windows of each subband every time constitute one 
block. A power spectrum is calculated for each block 
in a spectrum analysis member 4, and from the spec- 
trum thus obtained a masking threshold is deter- 
mined in member 5. After determining the masking 
threshold one is able to know what the maximum sig- 
nal power is which can be added to the audio signal 
of a subband in said time window. Below the masking 
threshold calculated for the audk) signal. DATA IN 
bits of the data signal are added. This is carried out 
so that a given number of consecutive bits of a data 
flow, e.g. three consecutive bits, form one word. Each 
word is interpreted to be an address representing a 
given sample value, in a three bit case there are 
therefore eight pieces of sample values. Thus, a sam- 
ple value represents in digital form a given voltage 
value (the power value of the audio signal). The se- 
lection of the word and of the sample value equivalent 
thereto is carried out in members. The sample values 
are grouped to be on the suitable sample windows of 
the subbands conforming to the equivalence of the 
sample value and the masking threshold of the sam- 
ple window of the subband in question, and summed 
into the audio signal samples of the subband in an ad- 
der 7. After being summed, the sample frequency of 
the signals of the subbands is Increased in member 
2, and the signals are again combined in a filter bank 
9 into a wideband audio signal, sounding in the eais 
of a listener just like the original audio signal although 
data information has been added thereto. 

Reception is in principle a reveree operation to 
transmisston. An audio signal in sample sequence 
form, received as In Fig. 2, is divided with filters 21 
into subbands. decimated in member 23 and grouped 
in member 23 into time windows T so that the same 
blocks are produced as in the transmission. The con- 
tents of a time window of a subband are analyzed in 
a spectrum analysis member 24. and thereafter the 
masking threshold can be determined thereon in 
member 25. After calculating said masking threshold. 
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the data information positioned below the nnasking 
threshold can be separated from the subband. Final- 
ly, the same address table is used in the block 28 as 
in the block 6 of the transmission, so that a conver- 
sk>n into bits can be made and by positioning the bits 
in succession the original bit flow is obtained. 

The encoding methods described here can be 
used for transmission of the surround sound on a 
transfer path intended for transferring a 2-channel 
stereo signal of the HDTV e.g. as follows: the original 
audio channels are indicated with letters L, R, C and 
S. The first two refer to a signal carried to the left and 
right loudspeaker, C refers to signals carried to the 
loudspeaker in the centre, and S to the numerous sig> 
nals (surround) to be carried to the loudspeakers on 
the sides of the listener. First, signals are mixed (ma- 
trixed) for instance for a new stereo pair as follows: 
L' = L + ^yfe (C + S) 
R' = R + (C + S) 

The surround information can be stored e.g. in 
signals 
Hi = C, and 
H2 = S. 

When next, reference being made to Fig. 1 and 
the description related thereto, a signal L' is carried 
to the audio input, a surround information signal Hi 
can be summed thereto as taught by the present 
method. The same applies to signals R' and H2. Here- 
by, a 4-channel stereo signal can be coded on a nor- 
mal stereo channel. 

In the receiver, the surround information hidden 
in the 2-channel stereo can be extracted and the 4- 
channel stereo signal L". R", C and S'' is obtained by 
removing the mixing (dematrixing): 
L" = L' - V2 (Hi + H2) 

R« = R'-l/2(H1 +H2) 

C" = Hi 
S" = H2. 

Unless the portion of the C and S signals hidden 
in the right and left channels is removed, the signal 
is not quite the same as in the original one, but, owing 
to the masking effect, the listener is not able to tell the 
difference. Since the format of the signal on the 
transfer path may. at least in theory, be made identi- 
cal with the format (e.g. NICAM) of the audio signal 
of the HDTV transfer path, the above signal is com- 
patible and the existing receivers can be used for re- 
peating the 2-channel stereo effect, or with the aid of 
additional circuits, for repeating the surround sound 
signal. 

The method known in the art and descrit)ed 
above involves certain drawbacks. First, both in the 
encoder of the transmitter and in the decoder of the 
receiver the hearing threshold utilizing the masking 
effect of the human hearing system must be calculat- 
ed using a model (PSM, Psycho Acoustic Model) 
modelling the masking of the human hearing system. 
The encoder and the decoder act therefore indepen- 



dently* Therefore, the decoders are elaborate and the 
receivers are expensh/e. Since in both the encoder 
and the decoder PSM is used in quantization, the 
quantization resolutions are different which leads to 
5 a distortion in the repeated audio signal, which is au- 
dible to the human ear and resembles the sound of 
water drops. This can be reduced by means of a better 
PSM but it raises the price of the receiver. In addition, 
because the information which can be added to a 
10 masking signal, varies continuously, it may in some 
instances be impossible to encode information to the 
extent required because when too much information 
is added, it results in distortk>n of the sound. The 
amount of the informatk>n to be encoded can be 
f5 somewhat raised using a complicated PSM, but that 
means rise in price. Thus, the method known in the 
art in a way restricts the use of the more sophisticated 
PSMs. Bit errors occur in the reception, caused not 
only by the transmission channel, but also by the.de- 
20 coding method used. 

The object of the present invention is to improve 
the method known in the art and described above so 
that the above drawbacks above are eliminated. The 
aim is according to daim 1 achieved so that an audio 
25 signal transmitted on a stereo channel and the infor- 
mation data hidden therein are separated using con- 
trol information transmitted by an encoder, being re- 
ceived on a separate channel. 

In the invention the information provided by an 
30 encoder of the system known in the art is made use 
of. This kind of information includes data on the data 
mode, information related to quantization, and infor- 
mation related to dematrixing. Said information is 
transmitted with a separate side channel simultane- 
55 ously together with the audio signals to a receiver 
which, controlled by the side-channel information, is 
able to process the 2-channel stereo signal it received 
and to convert it, e.g. into a multichannel sound sig- 
nal. The decoder of the receiver operates therefore 
AO controlled by the encoder of the transmitter, i.e. as a 
slave decoder. 

The invention is demonstrated with the aid of the 
accompanying schematic figures, in which 

Fig. 1 presents an encoder used in the method 
45 known in the art. 

Fig. 2 presents the decoder known in the art, and 
Fig. 3 shows in principle the method of the inven- 
tion. 

Figs 1 and 2 are the same as in the method of 
50 state of art and they have already been descrik)ed 
above. Fig. 3 presents a decoding block indicated by 
reference numeral 31 , which in its essential compo- 
nents can be simflar to the encoding block of prior art 
shown in Fig. 1. The encoder combines the enterin 
55 multichannel audio signal into a combined stereo sig- 
nal and "hides" a data signal therein with the akJ of 
the masking effect. Information on the data mode, 
quantization, and matrixing information are obtain- 
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t^^i^ ^ "'"strates the 

the Jransfer capadty of the hidden dala. These in- 
elude e.g information that no signal is included In 
certam channels relative to the level of the other 

t^y^' "H^^^^ '^^""^^ being de- 

,^!tH ^"^""^t^'*- the whole, the mode con- 
Tr^ P;;°<»ss.ng Of the special cases required in 
encoding the signal as faras they am notinduded in 
the sphere crf the normal mixing. The quantization 
data gn^es information on the quantization steps of 
t he masking signal and t he signal to be masked (to be 
hKlden) and on the number of bits, as well as on the 

^InT '"formation on how the original mult? 
Channel audio signal has been downmbced. In brief, all 

whk:ht«T^- ' 

wh«h the decoding can be carried out. The combined 

has been -hidden". Is adapted into an audk, channel 

1 «r .""''^ •^nsmi^ion on 

l^^-^'T S'?""*«"«>"sly the above listed data re- 
quired in decoding are transmitted on a separate low- 
speed digital Channel. If the data to be hiSden InTn 
audio Channel cannot at a certain moment of time be 
rece,ved.nanaudiochannel.l.e. the "masking effect- 

of said data can take place on said separate data 
channel the information transmitted whereby could 
be ^lled sHle-information because it Is tranLitted 
by the side of the actual audio channel 

In the receiverthe decoder 32 receives the audio 
channe signal and the side informatfon of the date 

ttontl ^1"'.'^"*'°*'^ by thedecoding Informa- 
t.on toansmjtted. ,t is enabled to decode the signal of 
the audio channel and to separate the date hidden 

In^T. , """"^ ""^^'"S ft ftrther 
enabled to generate e.g. a multfchannel sound signal 
The invention involves a numberof advantages In 
companson with the method of prior art. The psycho- 
acoustic modelling (PSM) is not needed in the decod- 
er of the receiver, the decoding and dematrixing be- 

In K f^^'- sophisticated PSM method 
can be used therein. In addition, a separate date 
channel enables date transmission also when the 
masking capacity of the audio signal is not suff fcient. 
Implemen^ng a decoder with silica is also consider- 
ably simplrfied. Since the decoder receives the quan- 
.zat«,n date from the encoder, no narrow ban3 Z 
tortion will be produced caused by faulty quantizatfon 
re.o,u.o„s. which is a great advantege as in the 
method known in the art distortion caused by faulty 
resolutions is a drawback which is distinctly audible 
sie fnf? '° "'"'^ transmission of 

kb/sisl„„ r T " ^''"^^^ «bat 16 

kb/s IS enough, and in additfon. date to be transmitted 
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thereon can be compressed and protected. Finally 

Zi^ f?*^ "'""^ '^'^^^ col 
trolled by the transmitter, the encoder oflhetransmit- 
ter may easily be modified without having to make 

S K"^tl'"*'^'^'^«"'^»'sP°^'btetoseleS 
freej^ Which date to transmit on an audio channel and 

Tau^ r L 's n^entioned be- 

cause the transmitter (i.e. the operator Of the system) 
controls the decoder. The system of the inve^on fe 
particularly well suited for transmission of the sur- 
IZh^"!" ^ ^''^ «hi«:h digitel 



Claims 

1. A method for combining an audk> signal and a 
date signal in an encoder and for transmissfon of 
the same on an audio channel, in which encoder 

- an audio signal incoming in sample se- 
quence form is divkled into subbands 

- a masking threshold Is simultaneously cal- 
culated in each subband for a sample dus- 
ter of equal size, the sounds below which 
are not audible to the human ear. 

- the biteof the samples remaining'belowthe 
masking threshold are replaced by the bits 
ofthe data signal, and 

- the subbands are combined, whereby a 
combined signal fortransmisston on the au- 
dio channel is obtained. 

characterized in that all the information is gath- 
ered in theencoderthat is required forseparating 
the combined signal again, and sakl Informatfon 
in the form of side information is simultaneously 
b^nsmitted on a separate date channel together 
with the combined signal. 

Method according to claim 1 
characterized in that merely the quantizatfon 
date ofthe audfo signal Is transmitted as the sfoe 
informatfon. 

Method according to claim 2, 
characterized in that said side informatfon con- 
tains the date concerning the data mode Illustrat- 
ing the arrangements for maximizing the amount 
of the hidden date, and the reconstructfon date 
of the date signal. 

Method according to daim 2. diaracterized in 
thattheside informatfon also contains part of the 
informatfon ofthe date signal. 

Method according to claim 2 

characterized in that the side informaUon also 

conteins the dematrixing date when the audfo 
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signal is a signal generated from a surround 
sound signal by matrixing. 

Method acconling to daim 1 for separating a 
^m^.ned audio signal and a data signal In a d^ 



coder, 
in which 
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15 



an audio signal entering in sample se- 
quence form is divided into subbands 
" !h««S!L '^^'"'"^ below the making 
^'^ ^P^'"*^'^ audio sig- 

- said separated bits are combined, whereby 
a data s^nal transmitted on the audio charv 
nel IS obtained, 

ch^cterizedinthatsuch information is received 
frrl'r'^t'''"*='^""«''"»''«^°''"<>^sideT^ 
formafon which Is required for separating the 

deLT ^ ^'S"^'- «'»'«^«by the 

7. Method according to daim 6 

TaT^m^'^j" '""'■^y '"^^ quantization 
ma«on ^ '"^'""'^ « '"fo'- 

«. Method according to daim 7 

characterized In that the information on the data 
mode and on the reconstruction the data signal 
's .nduded in the side information received 

9. Method according to daun 7 
characterized In that the received side informa- 

ZZ,7T^'- -format J Z 

cerning the data signal. 

10. Method according to daim 7 
diaracterized In that when the audio signal is a 
signal generated by matrixing from a multichan! 
nel sound signal, also the dematrixing infoo^^. 
tK,n IS included in the side information 

'\';X"cfa::r"~^'°--"-»^« 

Characterized in that they are used fortransmis- 
s.on of a surround sound signal in a tele^^n 
system in whid, digital sound transfer is uted 
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